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This  grant  provided  suppoii  to  8  graduate  students^  Of  these.  4  have  received  their  Ph.D.’s  and 
the  other  4  will  have  graduated  by  Tall  of  1987.  With  so  many  graduate  students  involved,  the 
research  pursued  has  necessarily  been  very  broad  within  optimization.- The  most  exciting^accom- 
plishment  is  a  new  trust  region  approach  to  global  convergence  for  nonlinear  programming  prob¬ 
lems.  Testing  has  also  begun  on  a  variable  metric  variant  of  the  Kamarkai  linear  programming 
algorithm  that  could  be  of  great  practical  significance  if  very  preliminary  tests  are  any  indication. 
Other  interesting  work  has  been  a  unified  convergence  analysis  for  the  many  variants  of  the  con¬ 
jugate  gradient  method,  a  convergence  analysis  of  the  popular  Nelder-Mead  algorithm,  a  novel 
use  of  interactive  computer  graphics  to  obtain  user  preferences  in  multi-objective  optimization,  a 
convergence  analysis  of  the  EM  algorithm  for  mixture  density  estimation,  and  a  survey  of  all  the 
•work  done  by  researchers  in  various  fields  on  nonlinear  programming  problems  in  which  some  sub¬ 
set  of  the  variables  always  appear  linearly.  vn- - 

The  original  proposal  was  mostly  concerned  with  ideas  developed  in  a  joint  paper  with  Dr.  Trond 
Steihaug  who  left  Rice  to  return  to  Norway  and  work  for  Statoil.  The  paper  will  appear  in  Siam 
Journal  for  Numerical  Analysis.  It  suggests  several  ways  to  attack  the  large  sparse  nonlinear  least 
squares  problem  by  column  grouping  schemes  connected  to  graph  coloring  which  are  generally 
used  to  cheaply  compute  large  sparse  Jacobian  approximations.  Our  results  were  interesting  but 
we  chose  not  to  follow  them  further  because  of  more  promising  avenues  of  research.  One  sidelight 
suggested  by  a  referee  for  the  paper  is  that  our  ideas  could  be  used  to  find  orderings  for  SOR  for 
large  linear  least  squares  problems.  Again  the  results  were  interesting  but  not  compelling. 

The  most  exciting  research  is  reported  on  in  the  joint  paper  with  Maria  Rosa  Celis  and  Richard 
Tapia.  We  found  a  conceptually  simple  and  elegant  way  to  get  around  the  main  problem  people 
have  encountered  in  trying  to  extend  trust  region  methods  to  constrained  optimization.  That 
problem  is  that  the  approximating  QP  might  not  be  feasible  (this  is  a  problem  with  the  SQP 
approach)  or  that  even  if  the  QP  is  feasible,  there  may  not  he  a  feasible  point  inside  the  trust 
region.  Our  way  around  both  these  difficulties  is  to  make  use  of  some  convergence  result*  for  the 
trust  region  algorithms  applied  to  finding  a  nonlinear  feasible  point.  Standard  theorems  show 
that  if  we  ask  for  some  percentage  of  at  least  as  much  linear  feasibility  in  the  Euclidean  norm  as 
would  be  given  by  the  steepest  descent  step  for  the  Gauss-Newton  quadratic  model  for  the  sum- 
of-squares  of  constraints,  then  we  will  converge  to  a  feasible  point  of  the  nonlinear  constraints 
with  a  standard  trust  region  implementation.  We  minimize  a  quadratic  model  of  the  Lagrangian 
subject  to  this  local  constraint.  An  interesting  benefit  of  this  approach  is  that  the  multiplier  of 
the  ‘sufficiently  feasible’  local  constraint  is  a  penalty  constant  for  the  augmented  Lagrangian 
which  can  then  be  naturally  available  as  a  merit  function. 

In  our  test  results,  we  found  that  this  ‘full’  trust  region  step  performs  very  well  in  the  majority  of 
cases,  but  sometimes  we  have  trouble  computing  the  step  with  our  modified  More-Hcbden-Rcinscb 
iteration.  This  has  motivated  us  to  seek  for  a  dogleg  implementation  of  the  above  idea  We  art 
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testing  such  an  algorithm  now. 

The  first  student  supported  by  this  contract  was  Professor  Richard  Hathaway,  currently  in  the 
Statistics  Department  at  the  University  of  South  Carolina  His  thesis  built  on  work  begun  in  my 
previous  ARO  contract  to  prove  that  a  ceriain  reasonable  regularization  of  the  mixture  density 
problem  made  it  well  posed  without  inhibiting  the  EM  algorithm  generally  used  for  practical 
applications. 

Dr.  Teresa  Parks  wrote  an  interesting  thesis  on  optimization  problems  in  which  some  of  the  vari¬ 
ables  can  be  eliminated  by  writing  a  closed  form  solution  for  them  at  the  optimum  in  terms  of  the 
other  variables.  The  simplest  example  is  the  so-called  VARPRO  or  separable  least  squares  prob¬ 
lem  in  which  one  has  a  nonlinear  least  squares  problem  with  some  linear  variables.  In  such  a 
case,  one  can  formally  eliminate  the  linear  variables  by  writing  them  in  terms  of  a  projection  that 
depends  on  the  nonlinear  variables.  This  gives  a  new  lower  dimensional  optimization  problem  to 
solve.  If  there  are  constraints  involved,  this  approach  can  backfire.  Simple  bound  constraints  on 
the  linear  variables  are  quite  common  in  such  problems  and  they  can  be  transformed  into  very 
nonlinear  constraints  on  the  remaining  variables  when  the  linear  variables  are  eliminated  Dr. 
Parks  found  many  connections  among  published  works  in  this  area  which  seem  not  to  have  been 
noticed  before. 

The  central  difficulty  with  the  Karmarkar  algorithm  for  practical  linear  programming  i«  the  solu¬ 
tion  of  a  large  sparse  linear  least  squares  problem.  We  are  currently  testing  a  BEGS  type  method 
for  updating  an  approximation  to  this  projection  matrix  in  order  to  cut  the  cost  of  this  part  of  the 
algorithm.  So  far,  we  have  only  test  results  for  a  single  small  problem,  but  they  are  excellent. 
We  are  currently  debugging  a  sparse  version  of  the  algorithm. 

We  have  gotten  some  very  interesting  results  on  conjugate  directions  methods  and  on  the  Neldcr- 
Mead  simplex  algorithm.  The  work  on  conjugate  directions  i;  based  on  a  ntw  and  simple  way  of 
viewing  the  large  number  of  algorithms  that  go  by  the  name.  We  prove  convergence  of  all  the 
methods  at  once  in  a  shorter  clearer  proof  than  is  generally  given  for  any  one  of  them.  This  work 
is  joint  with  Kathryn  Turner,  a  graduate  student  supported  by  this  grant,  who  should  finish  her 
degree  this  May. 

The  work  on  the  Nelder-Mead  algorithm  is  joint  with  Dr  Dan  Woods,  who  received  his  degree 
with  support  from  this  grant  in  May  ’85.  We  are  still  improving  our  results,  but  they  constitute 
the  first  convergence  results,  both  positive  and  negative,  ever  obtained  for  this  much  used  algo¬ 
rithm 

The  heavy  usage  of  this  algorithm  in  practice  would  be  reason  enough  to  analyst  it.  but  my 
interest  is  heavily  influenced  by  my  interest  in  exploring  the  use  of  such  pattern  search  algorithms 
in  parallel  optimization  Vtrginia  Torczon  will  explore  this  area  for  her  Ph.I).  thesis. 

Karen  Williamson  made  progress  in  developing  a  parameter  estimation  routine  for  inverse  prob¬ 
lems  for  ODE's  She  has  now  begun  to  consult  with  one  of  our  chemical  kinetics  researchers  on 
some  meaningful  lest  problems 

The  final  graduate  student  being  supported  by  this  grant  is  Michael  Lewis.  Mr.  Lewis  has  a  very 
strong  background  in  traditional  PDE’s  which  he  gained  at  lire  University  of  Minnesota  \\  e  have 
been  considering  the  infinite  dimensional  Broydcn  method  approach  suggested  by  Griewank  and 
Kelley  and  Sadis.  This  is  a  very  promising  approach  to  solving  nonlinear  dificrenlial  equations 
numerically. 
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